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OBJECTIVE — To create surveillance algorithms to detect diabetes and classify type 1 versus 
type 2 diabetes using structured electronic health record (EHR) data. 

RESEARCH DESIGN AND METHODS— We extracted 4 years of data from the EHR of a 
large, multisite, multispecialty ambulatory practice serving —700,000 patients. We flagged possible 
cases of diabetes using laboratory test results, diagnosis codes, and prescriptions. We assessed the 
sensitivity and positive predictive value of novel combinations of these data to classify type 1 versus 
type 2 diabetes among 210 individuals. We applied an optimized algorithm to a live, prospective, 
EHR-based surveillance system and reviewed 100 additional cases for validation. 

RESULTS — The diabetes algorithm flagged 43,177 patients. All criteria contributed unique cases: 
78% had diabetes diagnosis codes, 66% fulfilled laboratory criteria, and 46% had suggestive pre- 
scriptions. The sensitivity and positive predictive value of ICD-9 codes for type 1 diabetes were 26% 
(95% CI 12-49) and 94% (83-100) for type 1 codes alone; 90% (8 1-95) and 57% (33-86) for two or 
more type 1 codes plus any number of type 2 codes. An optimized algorithm incorporating the ratio of 
type 1 versus type 2 codes, plasma C-peptide and autoantibody levels, and suggestive prescriptions 
flagged 66 of 66 (100% [96-100]) patients with type 1 diabetes. On validation, the optimized 
algorithm correctly classified 35 of 36 patients with type 1 diabetes (raw sensitivity, 97% [87- 
100], population-weighted sensitivity, 65% [36-100], and positive predictive value, 88% [78-98]). 

CONCLUSIONS — Algorithms applied to EHR data detect more cases of diabetes than claims 
codes and reasonably discriminate between type 1 and type 2 diabetes. 



Electronic health records (EHRs) are 
transforming public health surveil- 
lance. Systems that can automati- 
cally extract, analyze, organize, and 
communicate EHR data to public health 
agencies increase the breadth, clinical 
detail, and timeliness of public health 
surveillance (1). EHR-based systems 
have rich potential to improve public 
health surveillance for diabetes, but little 
work has been done thus far to character- 
ize the accuracy of raw electronic data for 
diabetes surveillance or to create custom 
algorithms to accurately distinguish be- 
tween type 1 versus type 2 diabetes. 

Accurate discrimination between 
type 1 and type 2 diabetes is critical given 
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the different pathophysiology, epidemi- 
ology, prevention, management, and 
prognosis of these two diseases (2-7). 
Traditional public health surveillance sys- 
tems either do not distinguish between 
these two conditions at all or rely on 
self-reports to make the distinction. In ad- 
dition, most existing public health sur- 
veys are too small to meaningfully track 
low-prevalence conditions such as type 1 
in general and type 2 in youth. 

We hypothesized that EHR data 
could substantially enrich diabetes sur- 
veillance by facilitating continuous eval- 
uation of very large populations and 
leveraging clinical data to distinguish 
between type 1 and type 2 diabetes. We 



therefore sought to develop EHR-based 
surveillance algorithms to detect and 
classify type 1 versus type 2 diabetes 
and then apply them to a live, prospec- 
tive, EHR-based surveillance system to 
test their performance. 

RESEARCH DESIGN AND 
METHODS 

Setting and data sources 

We used retrospective EHR data from 
Atrius Health to develop diabetes surveil- 
lance algorithms. Atrius Health is a large, 
ambulatory, multisite practice based in 
Eastern Massachusetts that provides pri- 
mary and specialty care (including pediat- 
rics, internal medicine, and endocrinology) 
to >700,000 patients of all ages using a 
single consolidated EHR (Epic Systems, 
Verona, WI). We extracted encounter 
data from the EHR on all patients seen in 
the practice between 1 June 2006 and 30 
September 2010. The extract included all 
diagnosis codes, laboratory test results, 
and medication prescriptions. We in- 
cluded all patients with at least one en- 
counter in the EHR in our analysis. 

Detection of diabetes 

We created and applied a surveillance 
algorithm for diabetes based upon Amer- 
ican Diabetes Association (ADA) labora- 
tory diagnostic criteria (8), suggestive 
medication prescriptions, and ICD-9 codes 
(Table 1). We required at least two instan- 
ces of ICD-9 code 250. xx in patients with- 
out supporting laboratory or prescription 
flags. We did not include metformin in the 
algorithm as it may be prescribed for in- 
dications other than overt diabetes, such 
as polycystic ovarian syndrome and diabe- 
tes prevention. 

Classification of type 1 versus 
type 2 

We then began an iterative process to 
distinguish between type 1 and type 2 
diabetes within the population of patients 
flagged by the general diabetes algorithm 
in Table 1 . We started with a "straw man" 
algorithm designed to coarsely divide the 
population into pools of patients more 
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Table 1 — Surveillance algorithm for 
diabetes 



Any of the following criteria at any time: 

L Hemoglobin A lc >6.5% 

2. Fasting glucose >126 mg/dL 

3. Prescription for insulin outside of 
pregnancy 

4. ICD-9 code 250. xx on two or more 
occasions 

5. Prescription for one or more of the 

following medications: 
glyburide, gliclazide, glipizide, glimepiride 
pioglitazone, rosiglitazone 
repaglinide, nateglinide, meglitinide 
sitagliptin 

exenatide, pramlintide 



likely to have type 1 and patients more 
likely to have type 2. We began with 
this preliminary algorithm in order to 
make chart reviews more efficient: type 

1 diabetes is so rare compared with type 

2 diabetes that random sampling of un- 
selected patients yields very few type 1 
cases. The straw man algorithm allowed 
us to enrich a study population with type 
1 diabetes. A case of type 1 was defined as 
any patient with ICD-9 250.xl or 250.x3 
on two or more occasions, a current pre- 
scription for insulin, and no prescriptions 
for oral hypoglycemics at any time. A case 
of type 2 was defined as any patient with 
ICD-9 250. xO or 250. x2 on two or more 
occasions or a prescription for an oral hy- 
poglycemic at any time. 

We randomly selected 210 charts 
classified by the straw man algorithm for 
review: 70 patients classified as type 1, 60 
classified as type 2, and 80 left unclassi- 
fied by the straw man algorithm. Charts 
were reviewed for diabetes using ADA 
diagnostic criteria and for diabetes type 
(8). We used the following rules to assign 
"true" diabetes type (applied sequen- 
tially): endocrinologist diagnosis if avail- 
able, never on insulin (classify as type 2), 
C-peptide negative or diabetes autoanti- 
bodies present (classify as type 1), cur- 
rently on insulin but prior history of 
prolonged treatment with oral hypoglyce- 
mic alone (classify as type 2), and nonen- 
docrinologist physician diagnosis. 

We then created a series of candidate 
algorithms based on ICD-9 code frequen- 
cies, laboratory test results, and sugges- 
tive prescriptions to optimize sensitivity 
for chart-confirmed type 1 diabetes while 
maintaining high positive predictive val- 
ues. Patients who did not fulfill algorithm 



criteria for type 1 diabetes were presump- 
tively classified as type 2. The study 
endocrinologist (E.E.) and internist (M. 
K.) created the candidate algorithms 
based on clinical knowledge of diabetes 
management practices. 

We calculated the sensitivity and 
positive predictive value of all candidate 
algorithms using inverse-probability 
weighting to correct for the sampling 
strategy. We generated 95% CIs for these 
estimates using Monte Carlo simulations. 
Specifically, we simulated the number of 
true type 1 and type 2 patients among 
reviewed charts for each sampling strata 
(straw man type 1, straw man type 2, and 
straw man unclassified) using multino- 
mial distributions and probabilities esti- 
mated from the observed data. We 
repeated this process 1 million times 
and derived 95% CIs from the resulting 
2.5 and 97.5 percentiles. Calculations 
were executed using SAS version 9.3 
(SAS Institute, Cary, NC). 

We then created a final algorithm 
with optimized sensitivity and positive 
predictive value for type 1 diabetes by 
combining the candidate algorithms with 
the highest positive predictive values 
using "or" statements. We focused the op- 
timized algorithm on sensitivity to type 1 
diabetes because even slight misclassifica- 
tion of type 1 patients as type 2 is sub- 
stantially magnified after weighting for 
the greater size of the type 2 population 
and therefore exerts considerable cost in 
net sensitivity for type 1 . 

Validation 

We validated the optimized algorithm 
within a live, prospective EHR-based sur- 
veillance system based in the same practice 
that contributed derivation data (1,9). The 
algorithm was implemented on 1 Decem- 
ber 2011 and retroactively applied to all 
data resident in the surveillance system 
(1 June 2006 through November 30, 
201 1). We applied the reference criteria de- 
scribed above to 100 hitherto unreviewed 
charts: 40 classified by the optimized algo- 
rithm as type 1 and 60 classified as type 2. 
We calculated the raw and population- 
weighted sensitivity and positive predictive 
values of the final algorithm. 

Comparison of diagnosis 
codes alone versus the 
optimized algorithm 

We compared the sensitivity and positive 
predictive value of ICD-9 codes alone 
versus the optimized algorithm for de- 
tection of type 1 diabetes. We compared 



algorithm performance by calculating 
95% CIs for the differences in algorithm 
performance using multinomial distribu- 
tions and Monte Carlo simulations with 1 
million repetitions per comparison in a 
manner analogous to the method de- 
scribed above for calculating 95% CIs 
for each candidate algorithm. 

RESULTS 

Diabetes criteria 

The diabetes criteria flagged 43,177 indi- 
viduals. Characteristics of the population 
and percentages of patients flagged by 
each criterion are shown in Table 2. On 
review, diabetes was confirmed in 298 of 
310 patients (95% positive predictive 
value after correcting for sampling). The 
12 false-positives included one clinician 



Table 2 — Characteristics of patients flagged 
hy the diabetes surveillance algorithm 
described in Table 1 



Frequency, n (%) 


Age-group 


0-19 years 


519(1.2) 


20-59 years 


18,169 (42) 


>80 years 


6,264(15) 


Sex 


Male 


20,852 (48) 


Female 


22,325 (52) 


Race/ethnicity 


Caucasian 


24,746 (57) 


African American 


5,286 (12) 


Asian 


1,538 (3.6) 


Hispanic 


1,268 (2.9) 




Method of detection 


Hemoglobin A lc >6.5 


27,959 (65) 


Fasting glucose 




> 126 mg/dL 


1,647 (3.8) 


Hemoglobin A lc 
>6.5 or fasting 




glucose >126 mg/dL 


28,492 (66) 


Prescription for 
insulin outside 




of pregnancy 


10,275 (24) 


Prescription for 

an oral hypoglycemic 


Prescription for 
insulin or oral 




hypoglycemic other 
than metformin 


20,013 (46) 


ICD-9 250.xx on two 
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coding error (ICD-9 code 250. xx used for 
diabetes screening rather than diagnosis), 
two EHR coding errors (oral glucose tol- 
erance test 2-hour results miscoded as 
fasting glucoses), three algorithm pro- 
gramming errors (prescriptions for "insu- 
lin syringes" parsed as prescriptions for 
insulin), four patients with gestational di- 
abetes mellitus (pregnancy not coded as a 
discrete data element in the EHR), one 
patient with impaired fasting glucose, 
and one patient with impaired glucose 
tolerance. Among the 298 patients with 
chart-confirmed diabetes, 193 had type 2 
diabetes, 102 had type 1 diabetes, 1 had 
diabetes secondary to recurrent pancreati- 
tis, and 2 could not be classified (prescribed 
insulin but insufficient clinical data in the 
EHR to clarify the underlying diagnosis). 
Two patients with latent autoimmune di- 
abetes in adults were included in the count 
of type 1 patients for the sake of simplicity. 

Straw man algorithm 

The straw man algorithm for classifying 
patients as type 1 versus type 2 flagged 
1,095 patients as type 1 and 29,410 as 
type 2. The remaining patients were not 
classified. Type 1 diabetes was confirmed 
in 57 of 70 randomly selected type 1 
patients (population-weighted sensitiv- 
ity, 32%; positive predictive value, 
81%). Type 2 diabetes was confirmed in 
54 of 60 randomly selected type 2 patients 
(population-weighted sensitivity, 76%; 
positive predictive value, 90%). Among 
80 patients left unclassified by the straw 
man algorithm, 7 had type 1 diabetes, 65 
had type 2 diabetes, 2 had gestational 
diabetes mellitus, 1 was being prescribed 
insulin but lacked sufficient data to classify 
diabetes type, and 5 did not have diabetes. 

Candidate algorithms 

Candidate algorithms to distinguish type 1 
versus type 2 diabetes are presented in 
Table 3 along with their performance rates 
and 95% CIs. ICD-9 codes for type 1 di- 
abetes performed variably. Requiring an ex- 
clusive history of type 1 codes alone had 
high positive predictive value for type 1 di- 
abetes (94%) but poor sensitivity (26%). 
Requiring at least one ICD-9 code for 
type 1 with any number of type 2 codes 
increased sensitivity to 90% but lowered 
positive predictive value to 41%. Requiring 
at least two ICD-9 codes for type 1 and any 
number of type 2 codes preserved sensitiv- 
ity at 90% but increased positive predictive 
value to 57%. Seeking a plurality of ICD-9 
codes for type 1 diabetes (i.e. , ratio of type 1 
to type 2 ICD-9 codes >50%) had 



intermediate sensitivity (63%) but high 
positive predictive value (94%). 

Algorithms based on medication pre- 
scriptions performed with similar vari- 
ability. Presumptively classifying patients 
as type 1 based upon a prescription for 
insulin had excellent sensitivity (95%) 
but poor positive predictive value 
(23%), indicating that many type 2 pa- 
tients are treated with insulin. A single 
false-negative accounted for the less than 
perfect sensitivity of insulin prescriptions 
for type 1 ; the patient was prescribed an 
insulin pump that was recorded in the 
EHR as free text rather than as a struc- 
tured prescription. Likewise, classifying 
patients without any history of oral hy- 
poglycemics as type 1 was very sensitive 
(95%) but positive predictive value was 
low (15%). Creating an exception for met- 
formin increased sensitivity to 98% but 
lowered positive predictive value to 11%. 
The one medication with reasonable posi- 
tive predictive value for type 1 was glucagon 
(sensitivity, 68%; positive predictive value, 
79%). Notably, prescriptions for urine ac- 
etone tests had outstanding positive pre- 
dictive value for type 1 (sensitivity, 33%; 
positive predictive value, 100%). 

Combining ICD-9 and medication 
criteria increased positive predictive val- 
ues. For example, a plurality of ICD-9 
codes and a prescription for glucagon 
had a sensitivity of 31% but a positive 
predictive value of 100%. Similarly, a 
plurality of type 1 ICD-9 codes and no 
history of oral hypoglycemic prescrip- 
tions had a sensitivity of 59% and a 
positive predictive value of 99%. 

Optimized algorithm 

We optimized an algorithm to maximize 
sensitivity and positive predictive value 
for type 1 diabetes by combining all the 
high positive predictive value candidate 
algorithms using "or" statements (Table 
4). The final algorithm sought patients 
with any of the following: a plurality of 
ICD-9 codes for type 1 diabetes and a pre- 
scription for glucagon, a plurality of ICD-9 
codes for type 1 diabetes and a negative 
history of prescriptions for oral hypogly- 
cemics other than metformin, a negative 
plasma C-peptide, positive diabetes auto- 
antibody tests, or a prescription for urine 
acetone test strips. The final algorithm 
flagged 73 patients, including all 66 pa- 
tients with type 1 diabetes (raw sensitivity, 
100% [95% CI 96-100]; positive predic- 
tive value, 90% [82-96]). Correcting 
for the sampling strategy yielded a net 
population-weighted sensitivity of 100% 



(100-100) and a positive predictive value 
of 96% (91-99). 

Comparison of diagnosis 
codes alone versus the 
optimized algorithm 

The optimized algorithm was more sen- 
sitive for type 1 diabetes compared with 
surveillance for patients with one or more 
type 1 diabetes ICD-9 codes and any 
number of type 2 codes (100% vs. 90%, 
10% absolute difference [95% CI 4.9- 
19]), surveillance for patients with two 
or more type 1 diabetes ICD-9 codes 
and any number of type 2 codes (100% 
vs. 90%, 10% absolute difference [4.9- 
19]), and surveillance for patients with 
type 1 diabetes ICD-9 codes alone and 
no type 2 codes (100% vs. 26%, 74% ab- 
solute difference [50-88]). The optimized 
algorithm's positive predictive value for 
type 1 diabetes was also high compared 
with surveillance for one or more type 1 
codes (96% vs. 41%, 55% absolute differ- 
ence [3 1-72]) or two or more type 1 codes 
(96% vs. 57%, 39% absolute difference 
[11-62]), but equivalent to surveillance 
for patients with type 1 codes alone and 
no type 2 codes (96% vs. 94%, 2% abso- 
lute difference [-4.9 to 7.6]). 

Validation and implementation 

The final algorithm was applied to a live, 
prospective surveillance system for vali- 
dation. A total of 100 hitherto unreviewed 
charts were evaluated: 40 random charts 
flagged as type 1 and 60 random charts 
flagged as type 2. Chart review confirmed 
35 of 40 patients flagged as type 1. Of the 
remaining five, four had type 2 diabetes 
and one had gestational diabetes mellitus. 
Notably, three of the four false-positive 
cases were type 2 patients with a plurality 
of type 1 ICD-9 codes; these were all 
longstanding type 2 patients managed for 
years on insulin alone and labeled as type 
1 diabetic in their physicians' notes. 
Among the 60 patients classified as type 
2,57 were confirmed on chart review. Of 
the remaining three, one had type 1 dia- 
betes, one had gestational diabetes melli- 
tus, and one was prescribed insulin but 
had insufficiently detailed notes to deter- 
mine diabetes type. All told, the final al- 
gorithm correctly identified 35 of 36 
patients with type 1 diabetes (raw sensi- 
tivity, 97% [95% CI 87-100]) and 57 of 
6 1 patients with type 2 diabetes (raw sen- 
sitivity, 93% [85-98]). Projecting from 
these samples to the full study population 
using inverse probability sample weight- 
ing yielded net sensitivity and positive 
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predictive values of 65% (95% CI 36-100) 
and 88% (78-98) for type 1 diabetes, and 
net sensitivity and positive predictive val- 
ues of 100% (99-100) and 95% (88-100) 
for type 2 diabetes. Figure 1 depicts a heat 
map of the prevalence of type 2 diabetes 
by zip code that was automatically gener- 
ated by our EHR-based live surveillance 
system using the final algorithm. 

CONCLUSIONS— This work demon- 
strates the feasibility and potential utility 
of EHR data for automated diabetes sur- 
veillance. Leveraging the full array of data 
captured by EHRs, including laboratory 
test results and prescriptions in addition 
to diagnosis codes, increases both the 
sensitivity and granularity of surveillance 
compared with diagnosis codes alone. 
Our integrated algorithm, including lab- 
oratory, prescription, and ICD-9 criteria, 
captured more patients with diabetes 
than any one criterion alone; diagnosis 
codes flagged 78% of the final population, 
laboratory tests flagged 66%, and pre- 
scriptions for insulin or oral hypoglyce- 
mics (excluding metformin) flagged 46%. 
A combination algorithm seeking patients 
with a plurality of type 1 diagnosis codes 
and suggestive prescriptions, diagnostic 
laboratory test results (negative plasma 
C-peptide or positive diabetes autoanti- 
bodies), or prescriptions for urine acetone 
test strips classified patients as type 1 versus 
type 2 reasonably well. 

The application of these surveillance 
algorithms to EHR data has the potential 
to provide timely, clinically detailed in- 
formation on large numbers of patients at 
low marginal cost per patient. Further- 
more, in contrast to claims-based surveil- 
lance, EHR data streams can also provide 
rich contextual data about patients, in- 
cluding their demographics, clinical pa- 
rameters (e.g., blood pressure, BMI, 
hemoglobin A lc , and lipids), medica- 
tions, patterns of care, and how things 
are changing over time. EHR-based sur- 
veillance systems capable of extracting, 
analyzing, and reporting these indicators 
to public health departments in almost 
real time are emerging (1). EHRs are likely 
to become increasingly important sources 
for public health surveillance as more 
practices adopt EHRs and vendors begin 
to add functionality for public health sur- 
veillance. The federal government's 
"meaningful use" incentives provide sub- 
stantial financial encouragement for both 
of these goals (10). 

Very few public health surveys cur- 
rently distinguish between type 1 and 
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Current Dataset: General_Population Change 

Last Updated: 2011-12-07 



Outcome: Type 2 Diabetes True 
Filters: 
Last Encounter. (0-1,1-2) 




Predefined advanced 

Berkshire 
Boston 
Cape Cod 
Central Mass 
Fall River 
Franklin 
Lawrence 
Lowell 

Marthas Vineyard 
Merrimack Valley 
Metropolitan 
Montachusett 
Nantucket 
New Bedford 
Northern Middlesex 
Old Colony 
Pioneer Valley 
Southeastern 
Springfield 
Worcester 



Figure 1 — Screenshot of the Electronic medical record Support for Public health (ESP) live EHR-hased public health surveillance and reporting system. 
The map depicts the prevalence of type 2 diabetes by zip code among Atrius Health patients in Eastern Massachusetts. The system automatically detects 
diabetes and classifies patients as type 1 versus type 2 using data refreshed and analyzed nightly using the algorithms described in this article. 



type 2 diabetes despite their very different 
demographics, risk factors, prevention 
strategies, and management pathways. 
The Canadian Community Health Survey 
imputes diabetes type based on self- 
reported age at diagnosis, pregnancy his- 
tory, and medication usage (11). Some 
U.S. states add an extra question on dia- 
betes type to the Behavioral Risk Factor 
Surveillance System (12). Both of these 
classifications, however, depend upon 
self-reports rather than clinical and labo- 
ratory data. Vanderloo et al. (13) 
proposed a claims-based classification 
scheme for children based upon age, eth- 
nicity, and medication history, but the age 
and ethnicity criteria limit the generaliz- 
ability of their algorithm for general pop- 
ulation surveillance. Finally, some 
research surveys, such as the SEARCH 
for Diabetes in Youth, gather detailed 



physiological data to help classify patients 
as type 1 versus type 2 (14). This approach 
provides invaluable insights but expense 
and complexity preclude its use for rou- 
tine public health operations. EHRs' clin- 
ically rich data and efficient coverage of 
large populations constitute an additional, 
attractive option to improve public health 
capacity to routinely track type 1 and type 
2 diabetes. 

The persistent sources of error in our 
algorithms provide a telling window into 
the limitations of EHR data for public 
health surveillance. Sources of error in- 
cluded physician miscoding (type 2 di- 
agnosis codes assigned to patients with 
type 1 diabetes, diabetes diagnosis codes 
used for screening rather than frank dis- 
ease, and prescription for insulin pump 
recorded as free text rather than as struc- 
tured data), EHR miscoding (oral glucose 



tolerance tests coded as fasting glucoses), 
and algorithm programming errors (pre- 
scription for "insulin syringe" incorrectly 
parsed as prescription for insulin and fail- 
ure to recognize some patients as pregnant, 
leading to misclassification of gestational di- 
abetes mellitus as frank diabetes). 

Imperfections of EHR data also ac- 
count for the large discrepancy in the 
performance of the classification algo- 
rithm in derivation versus validation. 
Algorithm sensitivity dropped from 
100% in derivation to 65% on validation. 
This decrease was entirely attributable to 
misclassification of a single case of type 1 
diabetes as type 2 diabetes. The patient in 
question was a 45-year-old woman with 
type 1 diabetes since age 11. The EHR 
data on record included 33 encounters 
with ICD-9 codes for diabetes but 27 of 
the 33 encounters were miscoded as type 
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2 diabetes. In theory, the algorithm could 
still have classified her as type 1 since she 
was prescribed urine acetone test strips 
(as well as insulin and glucagon). These 
prescriptions were not recorded in the 
EHR prescription fields, however, be- 
cause they were provided by an endocri- 
nologist outside of our study practice. 

The marked drop in sensitivity from a 
single misclassification demonstrates the 
challenge of electronic surveillance for 
rare outcomes in large populations. The 
raw sensitivity of our algorithm on chart 
review appeared very good (35 of 36, 97% 
[95% CI 87-100]), but correcting for the 
sampling strategy multiplied the effect of 
the missing case many-fold. Inverse prob- 
ability sample weighting increased the es- 
timate of missed type 1 cases by a factor 
proportionate to the size of the type 2 
population. The net population- weighted 
sensitivity of the final algorithm for type 1 
diabetes therefore dropped to 65% (36- 
100). 

In addition, this missed case high- 
lights the ongoing risk of misclassification 
in surveillance studies using EHR data 
due to miscoding and the balkanization of 
patients' data between different EHRs that 
do not participate in data exchange. Two 
factors, however, may mitigate these 
sources of error over time. First, our live 
EHR-based surveillance system reclassi- 
fies patients anew every time fresh EHR 
data are uploaded. It is possible that this 
false-positive case may be reclassified cor- 
rectly in the future if her EHR-affiliated 
providers begin coding more accurately 
or enter a suggestive prescription. Like- 
wise, health information exchanges that 
consolidate data from multiple EHR sys- 
tems across a region may help diminish 
missing data (15). A recent survey of U.S. 
hospitals found that 10.7% currently ex- 
change data with unaffiliated providers 
(16). Current federal funding initiatives 
for states to develop regional information 
exchanges and meaningful use incentives 
for clinicians to exchange data are likely to 
encourage further growth of electronic in- 
formation sharing between unaffiliated 
organizations in years to come (17). 

Limitations of our study include the 
risk of incorrect reference classifications 
and algorithm overtraining. Our refer- 
ence standard classifications were made 
using retrospective chart review data 
rather than comprehensive physiological 
studies. We attempted to optimize refer- 
ence classifications by setting out a hier- 
archy of classification rules that gave 
greatest weight to endocrinologist diagnoses, 



consistent laboratory studies, and sugges- 
tive medication prescription histories 
rather than accepting any diagnosis from 
any clinician at face value. Nonetheless, 
misclassifications could have occurred 
due to endocrinologist error, false-positive 
or -negative laboratory test results, incom- 
plete clinical histories and prescription 
records, or primary care physician errors. 
Laboratory data in particular need to be 
interpreted with caution; some type 1 
patients, for example, can have negative 
antibodies and others may have remnant 
(3-cell function reflected in detectable 
C-peptide on standard clinical assays 
(18). Fewer than 5% of our classifica- 
tions, however, were based upon C-peptide 
or autoantibody measures. It would be 
informative to validate and possibly re- 
fine our suggested algorithm for type 1 
versus type 2 diabetes using data from 
studies with comprehensive physiologi- 
cal measurements on all patients (al- 
though even these studies sometimes 
suffer from controversy regarding their 
classification schemes). In addition, our 
classification algorithm is heavily depen- 
dent upon clinicians' coding, testing, 
and prescribing decisions. Even if the al- 
gorithm is optimized using physiologi- 
cal data as reference standard, there is 
still a future risk of misclassifying pa- 
tients if clinicians make incorrect diag- 
noses and then code and prescribe 
accordingly. 

The algorithms in this study were 
developed using data from a single EHR 
serving a single practice. Algorithm per- 
formance may be different if implemented 
in different settings with different clinical 
practice and coding patterns (19). The al- 
gorithm attempts to mitigate this risk by 
allowing multiple criteria for type 1 di- 
abetes, thereby increasing the probabil- 
ity that local practice and coding styles 
will still trigger the algorithm. The algo- 
rithms need to be validated in new set- 
tings, however, to test their robustness 
and portability. 

We created the algorithms in this 
paper for EHR-based surveillance systems 
but they may also be useful for the 
analysis of claims-based datasets. Claims 
data typically include many of the same 
elements as EHRs, including diagnosis 
codes, prescriptions, and laboratory tests, 
but usually only indicate that a test was 
performed, not the specific result. 
A major advantage of claims datasets, 
however, is that they include data on all 
of each patient's encounters, not just 
those within a given practice. This feature 



could mitigate the problem of missing 
data due to patients seeking care from 
multiple practices with unconnected 
EHR systems. Many of the candidate al- 
gorithms in Table 3 are suitable for appli- 
cation to claims datasets already insofar as 
they only include diagnosis codes and/or 
medication prescriptions. In addition, we 
assessed two potential adaptations of our 
final algorithm suitable for claims data. In 
the first adaptation, we retained the full 
algorithm but simply looked to see 
whether C-peptide and/or diabetes auto- 
antibody tests were performed rather than 
requiring a specific result. This preserved 
sensitivity but lowered positive predictive 
value for type 1 diabetes (100 and 79%, 
respectively). In the second adaptation, 
we removed the C-peptide and diabetes 
autoantibody criteria altogether. This de- 
creased sensitivity for type 1 diabetes but 
preserved a high positive predictive value 
(82 and 95%, respectively). 

In sum, we demonstrate the utility of 
EHR-based algorithms to detect and clas- 
sify patients with diabetes. Integration of 
these algorithms into EHR-based surveil- 
lance systems could provide a rich re- 
source for automated public health 
surveillance, practice management, and 
patient recruitment for clinical studies. In 
addition, we identify current sources of 
misclassification that limit algorithm per- 
formance. Knowing these sources of error 
could help practices and/or public health 
agencies design strategies to improve data 
quality and/or further refine our algo- 
rithm to mitigate these errors. Further 
work validating, enhancing, and applying 
these algorithms to public health surveil- 
lance systems is merited. 
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